Method of routing network traffic

ABSTRACT

A network switch includes an input layer to receive a data stream with a set of cells. Each cell includes data and a header to designate a destination device. The input layer includes a set of input layer circuits. A selected input layer circuit of the set of input layer circuits receives the data stream. The selected input layer circuit includes a set of queues corresponding to a set of destination devices. The selected input layer circuit is configured to assign a selected cell from the data stream to a selected queue of the set of queues. The selected queue corresponds to a selected destination device specified by the header of the selected cell. An intermediate layer includes a set of intermediate layer circuits, each intermediate layer circuit has a set of buffers corresponding to the set of destination devices. A selected intermediate layer circuit of the set of intermediate layer circuits receives the selected cell and assigns the selected cell to a selected buffer corresponding to the selected destination device. An output layer includes a set of output layer circuits corresponding to the set of destination devices. A selected output layer circuit of the set of output layer circuits stores the selected cell prior to routing the selected cell to a selected output layer circuit output node.

BRIEF DESCRIPTION OF THE INVENTION

[0001] This invention relates generally to high bandwidth datacommunications through computer networks. More particularly, thisinvention relates to an output queued switch with a parallel sharedmemory.

BACKGROUND OF THE INVENTION

[0002] As computer network traffic increases, there are ongoing demandsfor improved network communication and switching. The advent of opticalcommunication links has accelerated the need for ultra-fast networkswitching technologies.

[0003] There are many switching fabrics available in the market todaythat can provide switching bandwidth from 250 Gbps to 512 Gbps. Most ofthese switching fabrics are crossbar architectures that can scale up toa couple of Tbps. Unfortunately, it is difficult to obtain bandwidthshigher than this in view of the complexity associated with a centralizedarbitration and scheduling algorithm. Furthermore, implementations ofconventional crossbar architectures require relatively large chipcounts, resulting in relatively expensive systems. While packet switchtechniques have been suggested, proposed designs have not beensufficiently robust to accommodate high-speed requirements.

[0004] In view of the foregoing, it would be highly desirable to providean improved switching fabric. In particular, it would be highlydesirable to provide a switching fabric that is readily scalable withrelatively low chip counts to achieve high Tbps speeds.

SUMMARY OF THE INVENTION

[0005] The invention includes a network switch apparatus with an inputlayer to receive a data stream containing a set of cells. Each cellincludes data and a header to designate a destination device. The inputlayer includes a set of input layer circuits. A selected input layercircuit of the set of input layer circuits receives the data stream. Theselected input layer circuit includes a set of queues corresponding to aset of destination devices. The selected input layer circuit isconfigured to assign a selected cell from the data stream to a selectedqueue of the set of queues. The selected queue corresponds to a selecteddestination device specified by the header of the selected cell. Anintermediate layer includes a set of intermediate layer circuits, eachintermediate layer circuit has a set of buffers corresponding to the setof destination devices. A selected intermediate layer circuit of the setof intermediate layer circuits receives the selected cell and assignsthe selected cell to a selected buffer corresponding to the selecteddestination device. An output layer includes a set of output layercircuits corresponding to the set of destination devices. A selectedoutput layer circuit of the set of output layer circuits stores theselected cell prior to routing the selected cell to a selected outputlayer circuit output node.

[0006] The invention also includes a method of routing network traffic.The method includes receiving a data stream with a set of cells, eachcell including data and a header to designate a destination device. Aselected cell of the set of cells is assigned to a selected queue of aset of queues within an input layer circuit. The selected cell specifiesa selected destination device. The selected queue corresponds to theselected destination device. The selected cell is routed to a selectedintermediate layer circuit within a set of intermediate layer circuits.The selected intermediate layer circuit includes a set of bufferscorresponding to a set of destination devices. The selected intermediatelayer circuit assigns the selected cell to a selected buffer of the setof buffers. The selected buffer corresponds to the selected destinationdevice. The selected cell is then sent to a selected output layercircuit within a set of output layer circuits. The selected output layercircuit corresponds to the selected destination device. The selectedoutput layer circuit stores the selected cell prior to delivering theselected cell to an output node.

[0007] Advantages of the invention include high speed, versatility, highefficiency and a relatively low chip count. Additionally, the inventionincludes optional features, such as Quality of Service, fault toleranceand the ability to manage a number of different communication protocols,including Internet Protocol (IP), Time-Division Multiplexed (TDM),Asynchronous Transport Mode (ATM) and others.

BRIEF DESCRIPTION OF THE FIGURES

[0008] The invention is described with reference to the Figures, inwhich:

[0009]FIG. 1 illustrates a switch according to an embodiment of theinvention.

[0010]FIG. 2 illustrates an exemplary data cell that is processed inaccordance with an embodiment of the invention.

[0011]FIG. 3 illustrates an input layer circuit according to anembodiment of the invention.

[0012]FIG. 4 illustrates an intermediate layer circuit according to anembodiment of the invention.

[0013]FIG. 5 illustrates an output layer circuit according to anembodiment of the invention.

[0014]FIG. 6 illustrates an integrated circuit for use in the switch ofFIG. 1 according to an embodiment of the invention.

[0015]FIG. 7 is a flowchart showing operation of the switch according toan embodiment of the invention.

[0016]FIG. 8 is a dataflow diagram showing the operation of anembodiment of the invention.

[0017]FIG. 9 is a data diagram showing data cells as sent to theintermediate layers for each master frame according to an embodiment ofthe invention.

[0018]FIG. 10 is a dataflow diagram showing the operation of anembodiment of the invention.

[0019]FIG. 11 illustrates an embodiment of the invention wherein theinput layer and output layer are distributed across a set of sharedmodules.

[0020] Identical reference numbers in the figures refer to identicalelements in the drawings.

DETAILED DESCRIPTION OF THE INVENTION

[0021] The invention is described with reference to specificarchitectures and protocols. This description is for illustration and tootherwise demonstrate a mode of practicing the invention. Thisdescription is not meant to be limiting. For example, reference is madeto Internet Protocol, but any packet protocol is applicable. Moreover,reference is made to chips that contain integrated circuits, while otherhybrid or meta-circuits combining those described in chip form are alsocontemplated. The exemplary embodiment is provided for a switch where Nis 48, but could be any other number consistent with switch technology(e.g., 64).

[0022]FIG. 1 depicts a network switch 100 according to an embodiment ofthe invention. The switch 100 includes an input layer 110 that isconfigured to receive data at the input ports 112 a-112 n. The data maybe in the form of a cell, which is a fixed sized data segment. The datamay also be in the form of a packet, which is a variable sized datasegment containing many cells. The switch 100 is coupled to line cardsin a router. In particular, the input ports 112 a-112 n are connected toone or more line cards. By way of example, the line cards receive packetdata from a number of external sources. The input layer 110 is made upof a number of input layer circuits 114 a-114 n. The input layercircuits 114 a-114 n are each respectively coupled to the input ports112 a-112 n.

[0023] Each input port 112 receives a serial stream of cells. FIG. 2shows an exemplary cell 210, which includes a header 220 and a payload230. The header 220 includes attributes of the payload, including thedestination port of the switch that the data is intended for and otherinformation. In an exemplary embodiment, the attributes include packetidentification, error correction coding, protocol type (i.e., IP, TDM,ATM), and the like. In some aspects of the invention, the attributesinclude features, such as priority, Quality of Service (QoS), unicastand broadcast, error conditions, and the like.

[0024]FIG. 3 illustrates the internal structure of an exemplary inputlayer circuit 114. The input layer circuit 114 receives a data packet atits input port 112. A sorting circuit 312 processes the cell header ofthe data packet by decoding its destination. The sorting circuit 312 maybe implemented using conventional techniques.

[0025] The input layer circuit 114 includes a set of queues 314 a-314 n.Each queue corresponds to an output destination port. Thus, if there areN output destination ports, N queues are required. Observe that queue314 a corresponds to a first output destination port, queue 314 bcorresponds to a second output destination port, and so forth.Preferably, each queue 314 holds at least N cells, where N is the numberof output destination ports.

[0026] As cells are received, the queues 314 a-314 n are progressivelyfilled. When a queue is full, the queue is transferred to a transposercircuit 316. The transposer circuit receives a serial stream of datapackets from a queue 314 and transposes the data packets into a set ofparallel data packets that are applied to output ports 318 a-318 n ofthe input layer circuit 114. Observe that the input layer circuit 114receives a serial stream of input data packets and produces a set ofparallel output data packets. Each parallel output data packetoriginates from a single queue, which is used to store data packetsintended for a single destination. As discussed below, the paralleloutput data packets are distributed across a parallel shared memory,which operates to balance the load of incoming data. The parallel outputdata packets are distributed across the parallel shared memory inregions of the parallel shared memory intended for a single destination,as demonstrated below.

[0027] In one embodiment of the invention there are 48 separate queues114, wherein each queue 114 holds 48 data packets. Full queues areserviced in a round robin manner, as tracked by the scheduler 320.Preferably, the scheduler 320 periodically services non-full queues toavoid unreasonable delays.

[0028] Returning to FIG. 1, the data packets from the input layer 110are delivered, in parallel, to the intermediate layer 120. Like theinput layer 110, the intermediate layer 120 is made up of a number ofcircuits 124 a-124 n, referred to as intermediate layer circuits.

[0029]FIG. 4 depicts the internal structure of an intermediate layercircuit 124. The circuit 124 includes N input terminals 410 a-410 ncoupled to a sorting circuit 412 that is configured to sort the incomingdata cells by destination. The sorting circuit 412 is similar to that ofthe input layer sorting circuit 312. The intermediate layer circuit 124also includes N buffers 414 a-414 n to store the incoming data cells.Each buffer 414 has a corresponding output destination. That is, eachbuffer 414 stores data packets for a single output port. For example,cells destined for output port 1 are stored in buffer 414 a, cellsdestined for output port 2 are stored in buffer 414 b and cells destinedfor output port N are stored in buffer 414 n. The buffers 414 a-414 nare progressively filled as cells are sorted by the sorting circuit 412.However, the buffers 414 a-414 n differ from the input layer queues in anumber of important ways.

[0030] First, cells are released from the buffers 414 a-414 n on acontinuous basis. That is, unlike the input layer queue which onlyreleases cells after a queue is filled, the buffers 414 do not waituntil they are filled before sending out cells. This ongoing release ofcells is not arbitrated or otherwise subject to a centralized controlmechanism.

[0031] A second distinguishing feature between the input layer and theintermediate layer is that the intermediate layer circuits do not havetransposer circuits. Transposer circuits are not required since thebuffers 414 are coupled to terminals that send cells to the output layeras needed.

[0032] A third distinguishing feature between the input layer and theintermediate layer is that the input layer circuits have a serial inputnode and N parallel output nodes, while the intermediate layer circuitshave N parallel input nodes and N parallel output nodes.

[0033] One embodiment of the invention has 48 buffers 414. The scheduler420 is used to release cells from the buffers 414 as they arrive. Thereis no communication between the individual intermediate layer circuits124. Instead, each intermediate layer circuit 124 observes a stricttiming protocol, as discussed below.

[0034] Returning to FIG. 1, the switch 100 also includes an output layer130. Like the other layers, the output layer 130 is made up of a numberof circuits 134 a-134 n. FIG. 5 depicts the internal structure of anoutput layer circuit 134. The circuit includes N input terminals 510a-510 n coupled to a transposer circuit 512, which is configured totranspose into a serial data stream data cells received on the N inputterminals. Since the output circuit 134 can receive N cells in parallel,the transposer circuit 512 transposes the parallel cells into an N-deepqueue 514 so that the cells can be transferred to the destination outputport 516 in a serial fashion. This is performed at the direction of acircuit scheduler 520.

[0035]FIG. 6 shows an exemplary integrated circuit 610 for use in theswitch 100. Since the architectures of the input layer circuits,intermediate layer circuits and output layer circuits are similar, oneaspect of the invention is that the same integrated circuit may be usedin each of the layers. The control logic associated with the circuit forthat particular layer is enabled and the control logic not associatedwith the circuit is disabled. The chip 610 includes input layer logic620, intermediate layer logic 630 and output layer logic 640. The chipalso includes a RAM 650 that is controlled by the enabled logic. The RAM650 is configured to form queues 314, 414 and 514, as shown above. Thecircuit 610 may be used to implement an input layer by activating theinput module logic 620, while deactivating the intermediate module logic630 and the output module logic 640. Similarly, the circuit 610 may beused to implement an intermediate layer by activating the intermediatemodule logic 630, while deactivating the input module logic 620 and theoutput module logic 640. Finally, the circuit 610 may be used toimplement an output layer by activating the output module logic 640,while deactivating the input module logic 620 and the intermediatemodule logic 630. Advantageously, this feature allows the invention tobe implemented with a single chip architecture.

[0036]FIG. 7 is a flowchart 700 showing operation of the switch 100according to an embodiment of the invention. An explanation is providedin conjunction with FIG. 8, which is a dataflow diagram showingoperation of the switch according to an embodiment of the invention.FIG. 9 illustrates a data diagram showing data cells as sent to theintermediate layers for each master frame in a round robin technique, asdiscussed in connection with FIG. 7.

[0037] The first processing step associated with FIG. 7 is to receivecells at an input port (step 710). For example, a given port 112 areceives cells C1-CN that are destined for output port 132 a. In step712, the sorter circuit 312 decodes the cell header and determines thatthe cells are destined for output port 132 a. The sorter circuit 312stores the cells C1-CN in the input queue 314 a, as shown in FIG. 8. Instep 716, the input circuit checks the queues to determine if any ofthem are full, and as an additional possibility, whether the data in anyqueue is older than a predetermined threshold. This operation may beperformed by the scheduler 320. In the case of a non-full queue that isto be serviced, dummy cells are inserted to fill the queue. When theinput circuit determines that the queue 314 a is full, processingproceeds to step 718. At step 718, the cells are transposed, by thetransposer 316, into a set of parallel cells. The cells are then routedto the intermediate layer 120 in parallel. This is accomplished, asshown in FIG. 8, where the cell C1 is sent to intermediate circuit 124a, the cell C2 is sent to intermediate circuit 124 b and the cell CN issent to intermediate circuit 124 n.

[0038] In step 720, the cells are received by the intermediate layercircuits 124 a-124 n and each respective sorter circuit 412 decodes thecell headers and determines that the cells are destined for output port132 a. The selector circuit 412 stores the respective cells in the inputqueue 314 a. For example, selector circuit 412 a receives and decodescell C1 and places cell C1 in buffer 414 a. The cells are then bufferedin parallel as shown in FIG. 8 until they make their way to the outputterminals 416 of the intermediate circuits. Observe that the cells arenow distributed across a set of intermediate circuits 124. However, ineach intermediate circuit, they are stored in a buffer 414 correspondingto the output port to which the cells are destined. In this example, thecells are stored in the first buffer of each intermediate circuit 124.

[0039] In step 722, the cells C1-CN are sent to the output layer.Specifically, they are sent to the output circuit 134 a because thecells are destined for output port 132 a. In step 724, the cells arereceived by the output layer circuit 134 a. The cells are received inparallel and the transposer circuit 512 transposes the cells and storesthem in the N-deep queue 514. In step 726, the cells C1-CN are sent outthe output port 132 a and the switch function is complete.

[0040] This procedure continues for the other cells as shown in FIG. 9,which is a data diagram showing data cells as sent to the intermediatelayers for each master frame in a round robin technique. In such atechnique, all the circuits receive a frame clock in addition to asystem clock. Additionally, the circuits are instructed atinitialization as to which time slot to use since the assignment of thetime slots is arbitrary and can even be assigned based on any identifiedfault conditions. The round robin technique is an adequate arbitrationtechnique although other techniques may also be used in accordance withthe invention.

[0041] The operation of the invention is more fully appreciated with anadditional example. FIG. 10 illustrates a switch 100 with an input layer110, an intermediate layer 120, and an output layer 130, where eachlayer 110, 120, and 130 has N=3 circuits. In this example, nine cells(C1-C9) are processed. Observe in FIG. 10 that input layer circuit 114 areceives cells C1, C2, and C3. The header of each of these cellsindicates that each cell should be routed to a first output port 132 a.Accordingly, the sorter 312 a places the cells in a first queue 314 a,which corresponds to the first output port 132 a. In a similar manner,the input layer circuit 114 b receives cells C4, C5, and C6. The headerof each of these cells indicates that each cell should be routed to asecond output port 132. Accordingly, the sorter 312 b places the cellsin the second queue 314 b, which corresponds to the second output port132 b. The cells C7, C8 and C9 are processed by input layer circuit 114c in an analogous manner.

[0042] Once a queue 314 of the input layer circuit is full, in thisexample when three cells arrive, the cells are distributed in parallelto the intermediate layer, as discussed above in connection with thetransposer 316. FIG. 10 illustrates cells C1, C2, and C3 being routed inparallel. FIG. 10 also illustrates the remaining cells C4-C9 beingrouted in parallel to the intermediate layer 120. This results in theintermediate layer 120 storing cells destined for each output port. Forexample, intermediate layer circuit 124 a stores cell C1 destined forthe first output port 132 a in a first queue 414 a. Cell C4, destinedfor the second output port 132 b is stored in the second queue 414 b,while cell C7, destined for the third output port 132 c is stored in thethird queue 414 c. The cells stored by intermediate layer circuit 124 awere received by three different input layer circuits and will be routedto three different output layer circuits. Thus, this example helpsillustrate the load balancing operation performed by the intermediatelayer 120.

[0043] Each intermediate layer circuit delivers cells to the outputlayer 130 as the cells arrive. Thus, FIG. 10 illustrates thatintermediate layer circuit 124 a sends cell C1 to output layer circuit134 a, cell C4 is sent to output layer circuit 134 b and cell C7 is sentto output layer circuit 134 c. Similarly, intermediate layer circuit 124b sends cell C2 to output layer circuit 134 a, cell C5 is sent to outputlayer circuit 134 b and cell C8 is sent to output layer circuit 134 c.Each output layer circuit 134 receives cells in parallel and loads theminto a queue 514, as shown in FIG. 10. Queue 514 a of output layercircuit 134 a stores the cells C1, C2 and C3 destined for output port132 a. Queue 514 b of output layer circuit 134 b stores the cells C4, C5and C6 destined for output port 132 b. Finally, queue 514 c of outputlayer circuit 134 c stores the cells C7, C8 and C9 destined for outputports 132 c.

[0044] The operation of the invention has now been fully described;attention presently turns to a discussion of various features andbenefits associated with the invention. The invention achieves flowcontrol through back-pressure feedback. Back pressure feedback reliesupon downstream conditions (e.g., a blocked queue at an output port) toalter a data header of an upstream cell (e.g., the data header for acell at the input layer 110). The subsequent flow of the upstream cellis then processed in accordance with the downstream information. Thistechnique is more fully appreciated in connection with FIG. 11.

[0045]FIG. 11 illustrates the switch 100 of the invention in a slightlydifferent form. In FIG. 11, the input layer circuits 114 a- 114 n of theinput layer are distributed across a set of port cards 1100 a-1100 n.The port cards 1100 a-1100 n also include the output layer circuits 134a-134 n. In this configuration, a port card, say port card 1100 a, hasan input layer circuit 114 a and a corresponding output layer circuit134 a. Electrical leads 1110 between an input layer circuit 114 a and acorresponding output layer circuit 134 a allow information to beconveniently passed between the output layer and the input layer.

[0046]FIG. 11 also illustrates a set of prior art line cards 1102a-1102N connected to the port cards 1100 a-1100 n. Each line card 1102includes an ingress queue 1104 and an egress queue 1106.

[0047] The circuit topology of FIG. 11 allows for the output layer torelay information back to the input layer regarding conditions in theswitch 100. For example, the output layer can count the depth of each ofits queues and provide a signal to the input layer identifying which ofits queues are above a threshold congestion position. This signal can begenerated by the scheduler 520 associated with each output layer circuit134. This back-pressure signal can be handled within the switch. Forexample, the signal can be received by the scheduler 320 of an inputlayer circuit 114. In this example, the scheduler 320 instructs thesorter 312 to toggle a ready bit in the cell header. In this way, theready bit can be used to convey inter-layer flow control information.Alternately, the back-presssure signal can be sent to one or more linecards 1102. In this embodiment, one or more line cards respond to thesignal by only releasing high priority data destined for the output portexperiencing congestion.

[0048] There are many variations on the foregoing technique. Forexample, when the free cell pointer of output module 134 a is runninglow, the output module 134 a can signal all of the intermediate layercircuits 124 a- 124 n to stop sending traffic to the output module 134a. This can be done with a one bit signal applied to the input layercircuit 114 a on the same port card 1100 a. The input module circuit 114a responds to the one bit signal by de-asserting the ready bit in allcells departing for the intermediate layer circuits 124. Theintermediate layer can identify the congested output module by observingwhich input layer circuit 114 a is de-asserting the ready bit. Basedupon this information, the intermediate layer stops transmitting cellsto the congested output module 134 a.

[0049] The switch of the invention can also be configured to supportvarious levels of quality of service (QoS). Quality of service is anoteworthy aspect of the invention since some forms of data (e.g.,voice) frequently take priority over other forms of data (e.g., e-mail).In one embodiment of the invention, the cell header includes anattribute to assign the cell to a particular priority level. In such acase, a QoS attribute would be present in the header, as shown in FIG.2. If the priority is high, then the cell is processed through theswitch 100 in an expeditious manner. One way this can be accomplished isby selecting queues 314 at the input layer 110 that meet a particularthreshold. For example, suppose a queue has a number j of high prioritycells, in view of this number of high priority cells, the cells of thequeue are released, even if the queue is not full. This expedites theprocessing of high priority cells. This may not be the most efficientway to handle the cells, but there is a trade-off between handling thehigh priority cells versus maximizing the performance of the switch.This is particularly true when a majority of the cells are low prioritycells. In such a case, the lost performance may be negligible, while theenjoyment of the sound or video quality to the user is maintained.

[0050] Other techniques may also be used to implement quality of serviceprovisions. For example, the intermediate layer 120 can count the depthof each of its queues 414 and report to the output layer 130 which ofits queues are above a threshold position. The intermediate layer couldalso report quality of service parameters for the queued data. This canbe a factor in generating a back-pressure signal that can be handled atother layers of the switch or sent to the line cards 1102. The line cardwould respond to the signal by sending only high priority data throughthe switch destined for the output port experiencing congestion.

[0051] The architecture of the invention results in fault-tolerantoperation. Observe that the input layer 110 includes a set of inputlayer circuits 114, the intermediate layer 120 includes a set ofintermediate layer circuits 124, and the output layer 130 includes a setof output layer circuits 134. This architectural redundancy results indistributed processing without a critical central failing point. In thecase of the failure of a component of the invention, there is adegradation in performance, but not a catastrophic failure. For example,in the case of the failure of an intermediate layer circuit, there arestill N-1 intermediate layer circuits available to process traffic.

[0052] Fault tolerance is incorporated into the switch using a number oftechniques. For example, the line cards can have primary and secondarycontacts to the input layer. Referring to FIG. 11, line card 1102 a canbe configured to include contacts to input port card 1100 a and anadjacent input port card (e.g., input port card 1100 b, which is notshown for the sake of simplicity). If one set of contacts fail, the linecard transfers data cells to the secondary contact. This featureprovides fault tolerance at the input layer 110.

[0053] When the failure is in the intermediate layer 120, the inputqueues in the input circuits can be reduced (e.g. to N-1) and the failedintermediate layer circuit can thereby be avoided, as previouslyindicated. Since N is an arbitrary number, the reduction in theavailable intermediate layer circuits can be handled gracefully byreducing the input queue depth by one on-the-fly without an interruptionin packet processing. Finally, when the failure is in the outputcircuit, the output port can be flagged as disabled and the cells arerouted to a different output port and the router adjusts its routingfunctions to accommodate the failure. In each of these cases, theperformance is simply degraded and flagged, but does not result inoverall switch failure.

[0054] The examples of the invention provided up to this point have beendirected toward unicast packet communication. A unicast packet has onesource and one destination. The switch 100 can also be used to implementmulticast packet communication. In multicast packet communication, apacket has one source and multiple destinations.

[0055] Multicast packet communication can be implemented with cellheader information. For example, the cell header can include a bit mapspecifying a set of destinations for a single cell. Preferably, theinput layer circuits 114 identify whether an incoming cell is amulticast cell. The input layer circuits 114 would typically assign arelatively low priority to multicast cells. At the intermediate layer120, each intermediate layer circuit 124 a is preferably configured toread the cell header for multicast attributes, replicate cells and storethem in multiple buffers 414. This operation can be implemented with thesorter 312 and scheduler 320. This causes the replicated cells to besent to multiple output circuits 134, resulting in a multicast message.In one embodiment of the invention, each output layer circuit 134 isconfigured to make copies of multicast cells where required for multipleegress line cards. This operation can be implemented using the sorter412 and scheduler 420.

[0056] The switch 100 is also configurable to support Time-DivisionMultiplexed (TDM) and Asynchronous Transfer Mode (ATM) or other protocoltraffic. That is, the switch 100 can be configured to switch and routedigital telephony signals, which cannot be delayed (i.e., they must beprocessed with a very high priority within the switch). For example, inone embodiment of the invention, a particular output layer circuit, say134 a, is devoted to carrying TDM traffic. This output layer circuit hasa corresponding dedicated intermediate layer circuit, say 124 a, toinstantaneously route traffic to the output layer circuit. If thedesignated output layer circuit and intermediate layer circuits areunderutilized, they can be used to carry best efforts traffic.Alternately, the intermediate layer 120 can be time-divided to carry TDMtraffic.

[0057] In the exemplary embodiment, the intermediate layer 120 operateswithout timing signals between the individual intermediate layercircuits 124. Instead, the intermediate layer circuits 124 areinitialized to a synchronized state. In particular, a training sequenceis applied to each of the input layer circuits 114. The trainingsequence arrives within a window of time bounded by a link skew signaland a synchronization skew signal. The intermediate layer 120 then waitsuntil the training sequence is received from the input layer circuits114. The bias points for the different buffers 414 are then noted andare subsequently utilized as cells are received in normal operation. Thebias point data insures that the intermediate layer circuits operate inan identical state.

[0058] The parallel-shared memory output queue architecture of theinvention has a number of benefits. For example, the invention has alarge aggregate bandwidth, yet can be implemented with relatively lowchip counts, which results in lower cost and power consumption. Therelatively simple design of the invention avoids a centralized arbitermechanism or other type of complicated scheduler.

[0059] Those skilled in the art will recognize any number of variationson the base architecture described in this document. For example, theinput layer circuits may be implemented to include a number of queues314 for each destination port. Each queue can then be assigned adifferent priority to receive traffic with a corresponding priority.Similarly, each output layer circuit can include a set of output layerqueues associated with different channels and classes of services.

[0060] The invention has been described including the best mode known ofpracticing the invention. Those skilled in the art will recognize thatmodifications can be make to the invention while remaining within theclaims defined below.

In the claims:
 1. A network switch, comprising: an input layer includingN input layer circuits, each input layer circuit including an inputlayer circuit input port and N queues corresponding to N outputterminals; an intermediate layer including N intermediate layercircuits, each intermediate layer circuit including N buffers positionedbetween N intermediate layer circuit input terminals and N intermediatelayer circuit output terminals; and an output layer including N outputlayer circuits, each output layer circuit having N output layer circuitinput terminals and an output layer circuit output port, said N outputlayer circuit input terminals corresponding to individual intermediatelayer circuit output terminals of said N intermediate layer circuits. 2.The network switch of claim 1, wherein each input layer circuitincludes: a sorting circuit to route incoming cells to one of Ndestinations, each destination of said N destinations having acorresponding queue within said input layer circuit; and a transposercircuit coupled to said N queues and said N output terminals, saidtransposer circuit being configured to transpose cells stored in said Nqueues for delivery to said N output terminals.
 3. The network switch ofclaim 1, wherein each intermediate layer circuit includes: a sortingcircuit to route incoming cells to said N buffers, said N buffersthereafter delivering said incoming cells to said N intermediate layercircuit output terminals.
 4. The network switch of claim 1, wherein eachoutput layer circuit includes: a transposer circuit coupled to said Noutput layer circuit input terminals, said transposer circuit beingconfigured to transpose data cells received at said N output layercircuit input terminals; and an output layer circuit queue coupled tosaid transposer circuit and said output layer circuit output port. 5.The network switch of claim 1, wherein: said output layer includes anoutput layer circuit configured to generate a back-pressure signalrepresentative of the status of said output layer circuit; and saidinput layer includes an input layer circuit configured to be responsiveto said back-pressure signal by selectively inserting flow controlinformation into a data cell.
 6. The network switch of claim 1, wherein:said intermediate layer is configured to identify a multicast demandsignal in a cell and thereafter replicate said cell to produce amulticast signal.
 7. A network switch, comprising: an input layer toreceive a data stream including a set of cells, each cell including dataand a header to designate a destination device, said input layerincluding a set of input layer circuits, a selected input layer circuitof said set of input layer circuits receiving said data stream, saidselected input layer circuit including a set of queues corresponding toa set of destination devices, said selected input layer circuit beingconfigured to assign a selected cell from said data stream to a selectedqueue of said set of queues, said selected queue corresponding to aselected destination device specified by said header of said selectedcell; an intermediate layer including a set of intermediate layercircuits, each intermediate layer circuit including a set of bufferscorresponding to said set of destination devices, a selectedintermediate layer circuit of said set of intermediate layer circuitsreceiving said selected cell and assigning said selected cell to aselected buffer corresponding to said selected destination device; andan output layer including a set of output layer circuits correspondingto said set of destination devices, a selected output layer circuit ofsaid set of output layer circuits storing said selected cell prior torouting said selected cell to a selected output layer circuit outputnode.
 8. The network switch of claim 7 wherein said selected outputlayer circuit includes circuitry to generate a flow control warningsignal for application to said selected input layer circuit.
 9. Thenetwork switch of claim 7 further comprising a line card connected tosaid selected input layer circuit, wherein said selected output layercircuit includes circuitry to generate a flow control warning signal forapplication to said line card.
 10. The network switch of claim 9 whereinsaid line card delivers only high priority cells to said input layercircuit in response to said flow control warning signal.
 11. The networkswitch of claim 7 wherein said selected output layer circuit includescircuitry to produce a flow control warning signal in response to outputlayer congestion, said flow control warning signal being applied to saidinput layer, which produces a flow halt signal within a cell header,said intermediate layer including circuitry to identify said flow haltsignal and alter the delivery of cells to said output layer.
 12. Thenetwork switch of claim 7 wherein said input layer includes circuitry toidentify cell priority values within cell headers.
 13. The networkswitch of claim 12 wherein said input layer alters delivery of cells inresponse to said cell priority values.
 14. The network switch of claim 7wherein said input layer is operative in a normal mode to deliver datacells to each of said intermediate layer circuits and is alternatelyoperative in a fault mode to deliver cells to a subset of saidintermediate layer circuits that remain operative.
 15. The networkswitch of claim 7 wherein said intermediate layer is operative in anormal mode to deliver data cells to each of said output layer circuitsand is alternately operative in a fault mode to deliver cells to asubset of said set of output layer circuits that remain operative. 16.The network switch of claim 7 wherein said intermediate layer includesmulticast circuitry to identify a multicast command signal within amulticast cell and replicate said multicast cell in response to saidmulticast command signal.
 17. The network switch of claim 7 wherein saidintermediate layer circuit includes a first set of buffers to processhigh priority traffic and a second set of buffers to process best efforttraffic.
 18. The network switch of claim 17 wherein said output layerincludes a first set of output layer circuits to process said highpriority traffic and a second set of output layer circuits to processsaid best effort traffic.
 19. The network switch of claim 7 wherein saidintermediate layer processes cells without communicating between saidintermediate layer circuits.
 20. The network switch of claim 7 whereinsaid set of intermediate layer circuits process cells in accordance witha link skew value and synchronization skew value.
 21. The network switchof claim 7 wherein said input layer, said intermediate layer, and saidoutput layer are formed on a single semiconductor substrate, saidnetwork switch being configurable to enable a first region of saidsingle semiconductor substrate selected from said input layer, saidintermediate layer and said output layer, while disabling two regions ofsaid single semiconductor substrate selected from said input layer, saidintermediate layer and said output layer.
 22. A network switch,comprising: an input layer including N input circuits each coupled to aninput port and having N buffers therein corresponding to N outputterminals; an intermediate layer including N intermediate circuits eachhaving N input terminals and N output terminals, where the first outputterminal of each input circuit is coupled to the first intermediatecircuit and the Nth output terminal of each input circuit is coupled tothe Kth intermediate circuit, where K={1, 2, . . . N}; and an outputlayer including N output circuits each having N input terminals and anoutput port, where the first output terminal of each intermediatecircuit is coupled to the first output circuit and the Nth outputterminal of each intermediate circuit is coupled to the Lth outputcircuit, where L={1, 2, . . . N}.
 23. The network switch of claim 22,wherein each input circuit includes: a sorting circuit coupled to theinput port and configured to sort incoming data cells by destination; Nqueues coupled to the sorting circuit and configured to store aplurality of data cells; and a transposer circuit coupled to the Nqueues and the N output terminals and configured to transpose the cellsstored in a selected queue and to selectively communicate the cells tothe N output terminals.
 24. The network switch of claim 22, wherein eachintermediate circuit includes: a sorting circuit coupled to the inputterminals and configured to sort incoming data cells by destination; Nbuffers coupled to the sorting circuit and configured to store aplurality of data cells; and Wherein the N buffers are respectivelycoupled to the N output terminals and configured to selectivelycommunicate the cells to the N output terminals.
 25. The network switchof claim 22, wherein each output circuit includes: a transposer circuitcoupled to the N input terminals and configured to transpose data cellsreceived on the N input terminals; and a queue coupled to the transposerand the output port and configured to selectively communicate the cellsto the output port.
 26. A network switch, comprising: an input layerincluding N input layer circuits, each input layer circuit including aninput layer circuit input port and N queues corresponding to N outputterminals; an intermediate layer including N intermediate layercircuits, each intermediate layer circuit including N buffers positionedbetween N intermediate layer circuit input terminals and N intermediatelayer circuit output terminals; and an output layer including N outputlayer circuits, each output layer circuit having N output layer circuitinput terminals and an output layer circuit output port, said N outputlayer circuit input terminals corresponding to individual intermediatelayer circuit output terminals of said N intermediate layer circuits;wherein said input layer, said intermediate layer, and said output layerare formed on a single semiconductor substrate, said network switchbeing configurable to enable a first region of said single semiconductorsubstrate selected from said input layer, said intermediate layer andsaid output layer, while disabling two regions of said singlesemiconductor substrate selected from said input layer, saidintermediate layer and said output layer.
 27. The network switch ofclaim 26, wherein each input layer circuit includes: a sorting circuitto route incoming cells to one of N destinations, each destination ofsaid N destinations having a corresponding queue within said input layercircuit; and a transposer circuit coupled to said N queues and said Noutput terminals, said transposer circuit being configured to transposecells stored in said N queues for delivery to said N output terminals.28. The network switch of claim 26, wherein each intermediate layercircuit includes: a sorting circuit to route incoming cells to said Nbuffers, said N buffers thereafter delivering said incoming cells tosaid N intermediate layer circuit output terminals.
 29. The networkswitch of claim 26, wherein each output layer circuit includes: atransposer circuit coupled to said N output layer circuit inputterminals, said transposer circuit being configured to transpose datacells received at said N output layer circuit input terminals; and anoutput layer circuit queue coupled to said transposer circuit and saidoutput layer circuit output port.
 30. A network switch, comprising: aninput layer including N input layer circuits, each input layer circuitincluding: an input layer circuit input port and N queues correspondingto N output terminals, a sorting circuit to route incoming cells to oneof N destinations, each destination of said N destinations having acorresponding queue within said input layer circuit, and a transposercircuit coupled to said N queues and said N output terminals, saidtransposer circuit being configured to transpose cells stored in said Nqueues for delivery to said N output terminals.
 31. The network switchof claim 30, wherein said transposer circuit transposes said cellsstored in said N queues for parallel delivery to said N outputterminals.
 32. The network switch of claim 30 further comprising anintermediate layer including N intermediate layer circuits, eachintermediate layer circuit including N buffers positioned between Nintermediate layer circuit input terminals and N intermediate layercircuit output terminals.
 33. The network switch of claim 32 furthercomprising an output layer including N output layer circuits, eachoutput layer circuit having N output layer circuit input terminals andan output layer circuit output port, said N output layer circuit inputterminals corresponding to individual intermediate layer circuit outputterminals of said N intermediate layer circuits.
 34. A network switch,comprising: an intermediate layer including N intermediate layercircuits, each intermediate layer circuit including N buffers positionedbetween N intermediate layer circuit input terminals and N intermediatelayer circuit output terminals, said N intermediate layer circuitsasynchronously receiving cells at said intermediate layer circuit inputterminals and asynchronously delivering cells to said intermediate layercircuit output terminals.
 35. The network switch of claim 34 whereineach intermediate layer circuit includes a sorting circuit to routeincoming cells to said N buffers, said N buffers thereafter deliveringsaid incoming cells to said N intermediate layer circuit outputterminals.
 36. A network switch, comprising: an output layer including Noutput layer circuits, each output layer circuit having N output layercircuit input terminals and an output layer circuit output port, eachoutput layer circuit asynchronously receiving cells at said N outputlayer circuit input terminals and producing a serial cell stream at saidoutput layer circuit output port.
 37. The network switch of claim 36wherein each output layer circuit includes: a transposer circuit coupledto said N output layer circuit input terminals, said transposer circuitbeing configured to transpose data cells received at said N output layercircuit input terminals; and an output layer circuit queue coupled tosaid transposer circuit and said output layer circuit output port, saidoutput layer circuit queue producing said serial cell stream.
 38. Amethod of routing network traffic, comprising: receiving a data streamof cells at an input layer, each cell of said data stream of cellsincluding data and a header to designate a destination device; routing aselected cell from said input layer to a selected intermediate layercircuit within a set of intermediate layer circuits, said routingincluding routing said selected cell to a specified buffer within saidselected intermediate layer circuit that corresponds to said destinationdevice of said selected cell; and delivering said selected cell fromsaid selected intermediate layer circuit to a selected output layercircuit within a set of output layer circuits, said selected outputlayer circuit corresponding to said destination device of said selectedcell.
 39. The method of claim 38 further comprising duplicating saidselected packet when said header specifies that said selected packet isa multicast packet.
 40. The method of claim 39 wherein duplicating isperformed at said selected intermediate layer circuit.
 41. The method ofclaim 38 wherein said routing includes routing said selected cell to adedicated high priority traffic intermediate layer circuit when saidheader specifies that said selected cell has a high priority.
 42. Amethod of routing network traffic, said method comprising: receiving adata stream with a set of cells, each cell including data and a headerto designate a destination device; assigning a selected cell of said setof cells to a selected queue of a set of queues within an input layercircuit, said selected cell specifying a selected destination device,said selected queue corresponding to said selected destination device;routing said selected cell to a selected intermediate layer circuitwithin a set of intermediate layer circuits, said selected intermediatelayer circuit including a set of buffers corresponding to a set ofdestination devices, said selected intermediate layer circuit assigningsaid selected cell to a selected buffer of said set of buffers, saidselected buffer corresponding to said selected destination device; andsending said selected cell to a selected output layer circuit within aset of output layer circuits, said selected output layer circuitcorresponding said selected destination device, said selected outputlayer circuit storing said selected cell prior to delivering saidselected cell to an output node.
 43. The method of claim 42 wherein saidrouting is initiated when said selected queue reaches a specified cellvolume level.
 44. The method of claim 42 further comprising duplicatingsaid selected cell when said header specifies that said selected cell isa multicast cell.
 45. The method of claim 44 wherein said duplicating isperformed at said selected intermediate layer circuit.
 46. The method ofclaim 42 wherein said routing includes routing said selected cell to adedicated high priority traffic intermediate layer circuit when saidheader specifies that said selected cell has a high priority.
 47. Themethod of claim 42 further comprising: generating a flow control signalat said selected output layer circuit; forming a flow control headersignal within a header of an incoming data cell in response to said flowcontrol signal; and processing said incoming data cell through saidselected intermediate layer circuit and said selected output layercircuit in accordance with said flow control header signal.
 48. Themethod of claim 42 wherein said routing includes routing said selectedcell to a selected intermediate layer circuit within a subset ofintermediate layer circuits that remain operative after one or moreintermediate layer circuits within an original set of intermediate layercircuits become inoperative.
 49. The method of claim 42 wherein saidsending includes sending said selected cell to a selected output layercircuit within a subset of output layer circuits that remain operativeafter one or more output layer circuits within an original set of outputlayer circuits become inoperative.
 50. The method of claim 42 whereinsaid sending includes sending said selected data cell from said selectedintermediate layer circuit without communicating timing information withother intermediate layer circuits within said set of intermediate layercircuits.